{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "a9343007",
      "metadata": {
        "id": "a9343007"
      },
      "source": [
        "# Groq Llama3-8b-8192 Agent\n",
        "This agent uses Groq's `llama3-8b-8192` model to answer questions about Groq technology.\n",
        "It demonstrates a structured approach with YAML configs, prompt templates, and result handling."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/DhivyaBharathy-web/PraisonAI/blob/main/examples/cookbooks/Groq_LPU_Powered_AI_Assistant.ipynb)\n"
      ],
      "metadata": {
        "id": "mBWaooyypsyD"
      },
      "id": "mBWaooyypsyD"
    },
    {
      "cell_type": "markdown",
      "id": "b36ff766",
      "metadata": {
        "id": "b36ff766"
      },
      "source": [
        "## Dependencies\n",
        "We'll install the required Groq Python SDK.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "id": "518e7a6a",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "518e7a6a",
        "outputId": "98149592-19d0-472a-c31b-aae763fc8b07"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[?25l   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/129.6 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K   \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m\u001b[90m━━\u001b[0m \u001b[32m122.9/129.6 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.6/129.6 kB\u001b[0m \u001b[31m2.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "!pip install --quiet groq pyyaml"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d8569a5f",
      "metadata": {
        "id": "d8569a5f"
      },
      "source": [
        "## Tools Setup\n",
        "Initialize Groq client and helper functions for prompt generation.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "id": "b59b0d09",
      "metadata": {
        "id": "b59b0d09"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import yaml\n",
        "from groq import Groq\n",
        "\n",
        "# Set API key (replace with your own or use environment variables)\n",
        "os.environ['GROQ_API_KEY'] = 'enter your key'\n",
        "\n",
        "# Initialize Groq client\n",
        "client = Groq()\n",
        "\n",
        "def run_groq_chat(prompt_messages, model='llama3-8b-8192'):\n",
        "    response = client.chat.completions.create(\n",
        "        model=model,\n",
        "        messages=prompt_messages\n",
        "    )\n",
        "    return response.choices[0].message.content\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9302e8e6",
      "metadata": {
        "id": "9302e8e6"
      },
      "source": [
        "## YAML Configuration\n",
        "Define model and prompt templates via YAML.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "id": "238b8f7f",
      "metadata": {
        "id": "238b8f7f"
      },
      "outputs": [],
      "source": [
        "yaml_config = '''\n",
        "model: llama3-8b-8192\n",
        "prompt_template: |\n",
        "  You are an expert AI assistant knowledgeable about Groq's technology.\n",
        "  Provide a detailed answer to the user's question.\n",
        "  Then summarize the key points briefly.\n",
        "\n",
        "  User question: \"{user_question}\"\n",
        "'''\n",
        "\n",
        "config = yaml.safe_load(yaml_config)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6d2ff6b1",
      "metadata": {
        "id": "6d2ff6b1"
      },
      "source": [
        "## Prompt Construction\n",
        "Fill in the prompt template with user input.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "id": "2d42e686",
      "metadata": {
        "id": "2d42e686"
      },
      "outputs": [],
      "source": [
        "def build_prompt(user_question):\n",
        "    prompt_text = config['prompt_template'].format(user_question=user_question)\n",
        "    messages = [\n",
        "        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
        "        {\"role\": \"user\", \"content\": prompt_text}\n",
        "    ]\n",
        "    return messages\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3c9448d3",
      "metadata": {
        "id": "3c9448d3"
      },
      "source": [
        "## Main Logic\n",
        "Run the agent on an example question.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "id": "c59933e4",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c59933e4",
        "outputId": "bad10728-fd20-4e0d-fded-c29fd987a9b7"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Agent response:\n",
            "Groq's Low-Precision Unified (LPU) technology is a proprietary architecture designed to accelerate artificial intelligence (AI) and machine learning (ML) workloads. LPU-powered models, also known as Groq Models, have several advantages over traditional Graphics Processing Units (GPUs) in specific applications:\n",
            "\n",
            "1. **Improved energy efficiency**: LPU is optimized for low power consumption, making it suitable for edge, mobile, and embedded devices where power constraints are common. This is particularly important for applications that require long-lived deployments, such as autonomous vehicles or IoT sensors.\n",
            "2. **Enhanced accuracy**: LPU's customized precision and data type selection enable better representational precision for numeric computations, resulting in improved model accuracy. This is particularly beneficial for tasks that require high-fidelity calculations, such as medical imaging or natural language processing.\n",
            "3. **Simplified software development**: LPU's unified architecture simplifies the development process for AI/ML developers. Groq Models provide a consistent programming model across different inference scenarios, allowing for easier model deployment and optimization.\n",
            "4. **Increased throughput**: LPU's optimized arithmetic units and pipelined architecture enable higher Throughput per Watt (TPW) compared to traditional GPUs. This translates to faster processing times and higher compute density.\n",
            "5. **Flexibility and scalability**: LPU-powered models can be deployed across various hardware platforms, from small, low-power devices to large data center clusters. This flexibility allows developers to choose the optimal deployment scenario for their specific use case.\n",
            "6. **Customization and specialization**: LPU's architecture can be customized for specific workloads, allowing for optimized performance and power consumption. This customization potential enables developers to create highly specialized AI/ML hardware that matches their specific requirements.\n",
            "\n",
            "In summary, Groq's LPU-powered models offer significant advantages over traditional GPUs in terms of energy efficiency, accuracy, software development simplicity, throughput, flexibility, and customization.\n",
            "\n",
            "Key points:\n",
            "\n",
            "* Improved energy efficiency and suitability for edge, mobile, and embedded devices\n",
            "* Enhanced accuracy for high-fidelity calculations\n",
            "* Simplified software development with a unified programming model\n",
            "* Increased throughput and compute density\n",
            "* Flexibility and scalability across various hardware platforms\n",
            "* Customization potential for specific workloads and applications\n"
          ]
        }
      ],
      "source": [
        "user_question = \"What are the advantages of Groq's LPU-powered models compared to traditional GPUs?\"\n",
        "prompt_messages = build_prompt(user_question)\n",
        "agent_response = run_groq_chat(prompt_messages, model=config['model'])\n",
        "print(\"Agent response:\")\n",
        "print(agent_response)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "304a52dd",
      "metadata": {
        "id": "304a52dd"
      },
      "source": [
        "## Output\n",
        "The agent provides a detailed answer followed by a brief summary.\n"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "language_info": {
      "name": "python"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}